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ABSTRACT 

In applications where leak rates of components or systems are evaluated against a leak rate requirement, the 
uncertainty of the measured leak rate must be included in the reported result. However, in the helium mass 
spectrometer leak detection method, the sensitivity, or resolution, of the instrument is often the only component of 
the total measurement uncertainty noted when reporting results. To address this shortfall, a measurement uncertainty 
analysis method was developed that includes the leak detector unit’s resolution, repeatability, hysteresis, and drift, 
along with the uncertainty associated with the calibration standard. In a step-wise process, the method identifies the 
bias and precision components of the calibration standard, the measurement correction factor (K-factor), and the 
leak detector unit. Together these individual contributions to error are combined and the total measurement 
uncertainty is determined using the root-sum-square method. It was found that the precision component contributes 
more to the total uncertainty than the bias component, but the bias component is not insignificant. For helium mass 
spectrometer leak rate tests where unit sensitivity alone is not enough, a thorough evaluation of the measurement 
uncertainty such as the one presented herein should be performed and reported along with the leak rate value. 

BACKGROUND 

In leak detector tests, it is common to specify the sensitivity of the leak detector unit or determine the minimum 
detectable leakage rate. These terms are often used inconsistently, and can be a source of confusion. The American 
Vacuum Society standard AVS 2.1 Rev 1973 * provides a procedure for calibrating mass spectrometer leak detectors 
(MSLD) that uses sensitivity to determine the minimum detectable leak rate. In contrast, ASTM standard E13 16-14^ 
defines “minimum detectable leakage rate” and “sensitivity of leak tesf ’ synonymously. In addition, basic 
procedures for calibrating and performing leak detector tests are described in the ASNT Leak Testing Handbook,^ 
various ASTM standards, and manufacturers’ publications,^ among other sources. Often the manufacturer 
specifications for a leak detector will include drift, however, it is usually applicable to the most sensitive range of an 
operating mode and not necessarily the range in which the leak test is being conducted. Unlike other measurement 
devices, a specification for the overall accuracy of leak detectors is not given and procedures for determining total 
measurement uncertainty for mass spectrometer leak detector tests are not common. 

Just as the measurement testing errors are important in pressure change leak tests, * the measurement errors, or 
measurement uncertainty, in a leak detector test are also important to understand and include in the reported leak 
rate. Pryor and Walker describe the measurement errors commonly associated with the pressure change leak test,® 
and in an analogous way, the measurement errors associated with a helium MSLD test are described herein. In both 
types of leak tests, reporting a leak rate without its associated uncertainty is incomplete. Furthermore, the leak rates 
cannot be properly evaluated with respect to the governing leak rate requirement in the absence of associated 
uncertainty. 

When a measurement is taken and read directly from a piece of instrumentation, the uncertainty of the measured 
value is associated solely with the device. For example, a pressure gauge with 1 psig graduated increments has a 
measurement uncertainty of 0.5 psig. However, for calculated values where the value is a function of multiple 
variables, y X 2 , xs . . .), the uncertainty of each component (xj, xi, X 3 . . .) contributes to the total measurement 


uncertainty which is reported. This summation of individual errors is known as the propagation of errors. Neglecting 
the error associated with the covariance of terms (e.g., xixj, X 2 X 3 , etc.), the total uncertainty (Oy) can be calculated as 
shown in Equation 1 below.^ 


Equation 1 



A method was developed to determine the measurement uncertainty of helium leak rates using a helium MSLD 
based on the propagation of errors. The method is specifically for dynamic helium MSLD leak tests ^ although 
portions are applicable to other leak tests performed with a MSLD. The measurement errors associated with the leak 
test are classified as either bias errors or precision errors. The bias, also referred to as systematic error, can be 
thought of as the off-set between the measured value and the actual value, while precision terms provide information 
regarding how well the measurement was taken. ^ The root-sum-square method, or RSS method, is then employed 
to combine the total bias contribution with the total precision contributions as shown in Equation 2 where Ot is the 
total uncertainty, op is the total bias contribution to the uncertainty, and Ojj is the total precision contribution to the 
uncertainty. 


Equation 2 
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METHOD DEVELOPMENT 

When using a helium MSLD to quantify the leak rate of a component or system, the addition of knowing the 
reliability of the measurement provides a complete description of the performance with respect to leak rate. The 
reliability is expressed as the uncertainty, however, uncertainty of MSLD rueasurements is not a comruon 
specification from the luanufacturer. To address this deficiency, a five step luethod was developed to calculate the 
lueasurement uncertainty. Through the propagation of errors technique, the method accounts for the uncertainty 
from the individual coruponents of the leak test that are used in detenuining the final reported leak rate. Each step of 
the method is described in the following sections. 


(a) Measurement Repeatability 



(b) Measurement Hysteresis (c) Measurement Drift Over l ime 




Figure 1: Graphical representations of measurement a) repeatability, b) hysteresis, and c) drift which are 
used along with resolution to characterize the uncertainty of a mass spectrometer leak detector. 

Step 1 - Characterization of the Helium Mass Spectrometer Leak Detector 

The first step is to characterize the helium MSLD with respect to the four measurement uncertainty components of 
resolution, repeatability, hysteresis, and drift, ft is unusual to find specifications for resolution, repeatability, 
hysteresis, and drift for a MSLD, so these must be determined experimentally. Once these are determined for a given 
MSLD unit, the values are valid until the next calibration cycle or until a different measurement range is needed (see 
Case 1 and Case 2 in the Application Section for an example). The resolution of the measurement is directly related 
to the displayed value, and is a precision component of uncertainty, ft represents half the range of the rightmost digit 
after the decimal that is not displayed. For a display of #.#E-05 scc/s the resolution is #.#5E-05 scc/s, likewise, for 
#.##E-07 scc/s the resolution is #.##5E-07 scc/s. Repeatability is the other precision component and is the 
“closeness of the agreement between the results of successive measurements”.® Repeatability is determined by 
placing a calibration standard in the test system and recording the resulting measurement. This process is repeated 


multiple times under the same conditions. The standard deviation of the group of measurements is calculated and 
then multiplied by the corresponding t-statistic from the Student’s t-distribution for a two-sided test with a 95% 
confidence level. A minimum of 3 1 measurements is recommended. It should be noted that the response of a MSLD 
to leak rates over multiple decades of measurement is not necessarily linear, therefore, the calibration leak standard 
selected for the leak test should be sized correctly. For example, an E-08 leak should not be used as the calibration 
for a test with expected leak rates in the E-04 range. To determine hysteresis, the MSLD is used to measure 
calibration standards over a range of leak rates. Typically the range is within a single decade of measurement and is 
representative of the expected leak test decade of measurement. First the standards are measured in order from 
smallest to largest. The order is then reversed and the standards are measured largest to smallest. The maximum 
difference between the ascending and descending points at each calibration standard leak rate is taken as the 
hysteresis bias error. The final component, drift, is another bias component which provides information on how well 
the MSLD continuously measures a given value for a duration of time. With the calibration standard in the test 
system and the MSLD operating, measurements are recorded over a time period that meets or exceeds the expected 
test duration. Both the measured leak rate value and the time at which the measurement was recorded are 
documented. Similar to hysteresis, the maximum absolute difference between the calibration standard value and the 
resulting measured values is the drift. Examples of visual representations of these components are given in Figure 1. 
Both the precision (omslD(|)) and bias (omsldp) of the MSLD are calculated using the RSS method to sum the 
individual precision and bias components. 

Step 2 - Characterization of the Calibration Standard 

After the MSLD is characterized, the next step is to characterize the calibration standard’s uncertainty. The 
uncertainty of the calibration standard and its temperature compensation are specified by the manufacturer or 
calibrating company. When temperature compensation is required, the uncertainty is determined using the 
propagation of errors (Equation 1). The uncertainty (Ocaistdp) is a bias error as it is an indicator of the degree to which 
the measured value could vary from the stated calibrated value. 

Calibration standards are classified as one of two types: reservoir standards or non -reservoir standards, which are 
also known as closed and open standards, respectively. In reservoir type standards, the helium supply is contained 
within the standard, whereas in the non-reservoir standards, helium must be supplied to the standard at the specified 
pressure with the specified purity of gas. Therefore, in addition to the uncertainty of the calibration standard itself, 
the uncertainty of the pressure measurement device(s) must be accounted for and included in the total uncertainty of 
the calibration standard. The pressure measurement device(s) also have a precision component of repeatability that 
must be tracked as the calibration standard leak rate precision uncertainty (ogstdcj)) and included in the propagation of 
errors for the correction factor (see Step 4). 

Step 3 - Characterization of the Leak Rate System 

The previous two steps can be applied to each individual leak test performed with a given test system or set-up. In 
this step. Step 3, the leak rate test itself is performed. Measurements are taken and recorded for each of the following 
test phases: pre-test calibration standard, leak rate test, and post-test calibration standard. The helium supply 
pressure is also taken and recorded when applicable. 

Step 4 - Calculate the Test Measurement Uncertainty 

Once the leak test has been completed, the data can be analyzed and the measurement uncertainty calculated. In this 
step, the correction factors, the reported leak rate, and all associated uncertainties are determined. 

Correction Factor Calculations 

In dynamic system leak testing, correction factors are used as part of the calibration of the leak test system. The 
calibration standard is attached to the system at the point furthest from the inlet to the MSLD. The MSLD measured 
leak rate is compared to the leak rate of the calibration standard, and a correction factor is used to adjust the 
measured leak rate to the calibration standard leak rate. A simple correction factor equation is K = Qstd/Qsm where K 
is the correction factor, Qstd is the calibration standard leak rate as stated by the manufacturer, and Qsm is the MSLD 
output (i.e., the measured leak rate). Often the correction factor is more complex than this and is based on correction 
factors taken at the beginning and end of a test. For the calculations reported here, Qstd is an open type calibration 
standard with a pressure transducer. The leak rate is calculated using the input helium supply pressure (P) and the 


manufacturer’s equation and pressure term coefficients (Equation 3). The correction factor K is the average of the 
initial and final correction factors, K, and K/ (Equation 4). 

Equation 3 Qstd = a 2 P^ + aiP + ao Equation 4 Ki = and Kf — 

Qsmi Qsmf 


Correction Factor Uncertainty Calculations 

The propagation of errors as stated in Equation 1 is used to calculate the bias and precision uncertainty components 
of Qstd and K,. Equations 5 and 6 are specific for the bias contribution to uncertainty. The precision contributions 
have similar equations where the uncertainty terms (o) are precision uncertainties. Recall that Ocaistdp is a bias term 
without a corresponding precision term. The MSLD uncertainty components calculated in Step 1 are the uncertainty 
terms applied to the measured leak rate of the standard; that is Omsldp = cfQsmp, and OmslD(|) = OQsmf Equations 5 and 6 
are written specifically for the pre-test calibration and the initial correction factor. The post-test calibration and the 
final correction factor equations are the same except the subscript i is replaced with the subscript f. 


Equation 5 


Qstdifi 
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Equation 6 






Reported Leak Rate Calculations 

How the correction factor is applied to the measured leak rate of the test is specified in the leak test procedure. For 
the leak test described here, the measured leak rate is multiplied by the correction factor to obtain the reported leak 
rate, Qr = Qm* K. 


Reported Leak Rate Uncertainty Calculations 

Equation 1 is used again to calculate the bias and precision uncertainty components of the reported leak rate. The 
RSS method. Equation 3, is used to combine the bias and precision errors into one total measurement uncertainty. 


Equation 7 


Equation 8 


Equation 9 



Step 5 - Report the Leak Rate with Associated Measurement Uncertainty 

The final step in the method is the reporting and evaluation of the leak rate. The measured leak rate is reported with 
its associated measurement uncertainty in the format of Leak Rate ± Total Measurement Uncertainty, Qr ± oqr. The 
leak rate is then evaluated for acceptance or rejection. When evaluating the leak rate, the associated uncertainty is 
added to the leak rate value for requirements with upper limits (i.e., the leak rate shall not exceed requirements) and 
subtracted from the leak rate value for requirements with lower limits (i.e., the leak rate shall be at least 
requirements). For example, when the requirement is such that the leak rate cannot be greater than 4.0E-08 scc/s, the 
reported leak rate 3.3E-08 scc/s ± 2.7E-09 scc/s evaluates 3.6E-08 scc/s against the requirement, and the system is 
deemed acceptable. 


METHOD APPLICATION 

Dynamic helium MSLD tests were performed on candidate seals for spaceflight hardware. The leak rate 
requireruents represented the maxiiuum allowable leak rate of habitable cabin air to space. Therefore, in order to 
properly evaluate the leak rates, the uncertainty in the reported value was required. The tests were performed using a 
MSLD with a sensitivity specification of 6E-10 scc/s. Two cases are presented (Table 1) to demonstrate the 


difference in measurement uncertainty while using the same MSLD unit and therefore the same sensitivity. The only 
difference in the uncertainty components of the MSLD is the resolution term (Table 1, Row 1) because the expected 
test measurement range is different for the two cases. The measurement uncertainty in Case 1 is 6.6% of the reported 
leak rate while the measurement uncertainty for Case 2 is 13.9% of the reported leak rate. In both cases, the 
precision contribution is greater than the bias contribution. The results highlight that sensitivity alone is not enough 
to convey information about the reliability of the measured and reported leak rate. 


Table 1: Helium Leak Test Results for Two Cases Using a MSLD 



Case 1 

Case 2 

MSLD hysteresis (P) 
MSLD drift (P) 

MSLD resolution ((()) 
MSLD repeatability ((()) 

OHyst = 0 scc/s 
ODrift = 0 scc/s 
ciResoiu = 5.00E-05 scc/s 
ORepeat = 9.42E-05 SCC/s 

OHyst = 0 scc/s 
ODrift = 0 scc/s 
OResoiu= 5.00E-07 scc/s 
ORepeat= 9.42E-05 scc/s 

Overall Bias 

Overall Precision 

Total Measurement Uncertainty 

Op = 7.67E-05 scc/s 
o,|, = 1.08E-04 scc/s 
oq = 1.33E-04 scc/s 

Op = 3.99E-06 scc/s 
0 (|, = 1.41E-05 scc/s 
Oq = 1.46E-05 scc/s 

Reported Leak Rate Q 

Q,- = 2.01E-03 scc/s ± 1.33E-04 scc/s 

Qr= 1.05E-04 scc/s ± 1.46E-05 scc/s 


SUMMARY 

To provide an indication of the reliability of the leak rate from a MSLD, a method for determining the total 
measurement uncertainty was developed for use in these tests. The method uses the concept of propagation of errors 
to account for the measurement uncertainty associated with the leak detector unit itself, the calibration standard, and 
the correction factor. The leak detector unit is characterized with respect to its resolution, repeatability, drift, and 
hysteresis instead of the sensitivity. In application, it was found that the precision component contributed more to 
the total uncertainty than the bias component, but the bias component was not insignificant. 
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